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STRONG  LAWS  OF  LARGE  NUMBERS  FOR  ARRAYS  OF 
ORTHOGONAL  RANDOM  VARIABLES1) 

By  F.  Moricz  of  Szeged 
and 

R.  L.  Taylor  of  Athens,  Georgia 
and  Chapel  Hill,  North  Carolina 

£ —  '  f 

§1. ''"Hilbert  Space  Valued  Random  Variables  ,  ■  >  . 

-'<yr  'f  jQr  c  ,  , 

Let  H  be  a  separable  Hilbert  space  with  inner  product  denoted  by 
(♦,*)•  By  a  rowwise  orthogonal  array  we  mean  a  set  0^:  k  *  1,2,...  ,n; 
n»  1,2,.,.}  of  random  variables  (in  abbreviation:  r.v.'s)  such  that 

(1.1)  a2  :  -  EC  IX  .  II2]  <  » 

nk  *-  nk  -1 

and 

(1.2)  EC(Xnk,  Xnj)]  -  0  (k  +  j?k,j  =  l,2 . .  n  -  1,2, . . .  ) . 


Details  on  basic  properties  and  measurability  considerations  for  Hilbert 
and  Banach  spaces  are  available  in  the  literative  cited  in  the  references. 

We  will  consider  the  means 

(1.3)  C  :  -  Z  X  .  (n  -  1,2,...) 

n  na  k-1  nk 


)  This  research  was  completed  while  the  first  named  author  was  a 
visiting  professor  at  the  University  of  Wisconsin,  Madison,  in  the  academic 
year  1985-86.  Also,  this  research  was  supported  in  part  by  the  Air  Force 
Office  of  Scientific  Research  under  Contract  No.  F  49620  85  C  0144  while  the 
second  named  author  was  at  the  Center  for  Stochastic  Processes,  University 
of  North  Carolina,  Chapel  Hill,  N.C. 
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where  a  is  a  fixed  positive  number.  Following  Hsu  and  Robbins  (1947)  we 


*'.•1 


say  that  the  sequence  {£  }  converges  to  zero  completely,  in  sign  liraC  -0 

n  n-~>  n 

completely,  if  for  every  e  >  0, 


s  PC«c  i  >  t]  <«. 

n«l  n 


By  virtue  of  the  Borel-Cantelli  lemma,  complete  convergence  implies 
almost  sure  (in  abbreviation:  a.s.)  convergence.  The  converse  is  not  true 


'yV 

!<('? 

V* 


in  general,  except  the  case  when  the  £  are  independent. 

First  we  present  a  simple  sufficient  condition  to  ensure  complete 
convergence . 


LJt 

m 


% 


't‘i; 

>  *  j.1 


Theorem  1 .  Let  (X  .)  be  a  rowwise  orthogonal  array  in  a  separable 
Hilbert  space  H.  If 


(1.4) 


1  n  2 
“  Z  a  .  <  « 
2a  nk 


n»l  ri  k-1 


for  some  a  >  0,  then 


if*'!' 

vy 

v.V 

yjv 

it 


(1.5)  limC  *0  completely 

n-*«  n  ^ 


Proof .  By  (1.1)  and  (1.2), 


*,V}( 

,'i'o 

Ml: 


(1.6)  £[»(#']  »irE[(E  X  ,  sx  )] 

n  nz  k-1  nk  j-1  nJ 


Z  Z  EC(X  .,  X  )D 
nzot  k-1  j-1  nk  nj 


Z  o  ,  . 
n2a  k-1  nk 


"V  i  )  *  V  ^  » "  •  .  i  *  4  "*,3  S  V  *  fc.*  4  ^  $  4  •  t  j  ^  ^  A  •  *“  ^  ^  ^  ^  ^ 


Hence,  by  (1.4), 


£  e  C  ii  c  « 2  □ 

n=l  n 


n 

Z 

k-1 


<0°  , 


This  implies  (1.5)  v  he  Chebyshev  inequality. 

A  simple  conse^  e  is  the  following. 

Corollary  1.  Let  {X^}  be  a  rowwise  orthogonal  array  in  a  separable 
Hilbert  space  H  such  that 


(1.7) 


o  i  <  o. .  ( n  =  k+1 ,  k+ 2 ,  • . . $  k  *  1,2,...). 

nk  -  kk 


If 

(1.8) 


Z  - 
k=l  k 


2a- 1 


< 


00 


for  some  a  >  i ,  then  we  have  (1.5). 

Proof.  In  fact,  (1.7)  and  (1.8)  imply  (1.4)  in  case  a  >  i  as  follows 


Z 

n»l 


1 

2a 


1 


n  2  ” 

Z  a  .<  Z  0 
k-1  nk  ”  n«l  n2a 


n 

Z 

k-1 


kk 


Z 

k-1 


kk 


Z 

n-k 


■h  -  *l> 


Z 

k-1 


kk 


2a- 1 


Remark  1.  The  weaker  condition 


(1.9) 


Z 

k-1 


<  00 


implies  only  the  complete  convergence  of  the  lacunary  subsequence 
(C2p:  P  *  0,1,...}.  Indeed,  by  (1.1),  (1.2),  (1.7)  and  (1.9), 
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Z  ECK  J21  •  2  1 


*0  22aP  k-1  2P’k 


p-o  2 


<  z  z 

~  p-o  22aP  k-1 


l L_  4  2 

2ap  k,,  °kk 


Z  Z  -s 

M  p:2P>k  2 


'» V 


0(1)  z  -fs  < 

k-1  k2a 


Hence,  the  Chebyshev  inequality  yields  the  complete  convergence  of  {C  }• 


*  t 

l.V? 

ki} 

1  ,«! 


k  J 


Now  the  surprising  fact  is  that  Theorem  1  is  the  best  possible 


(i)  for  real  valued  (H  ■  R)  r.v.'s;  and  even 
(ii)  if  we  require  orthogonality  not  only  within  each  row  but  between 


VS 

sifv 

‘4 

V-* 

J 

'M 


any  two  rows  in  the  array  (X  .}. 


Theorem  2.  Let  {onk>  be  an  array  of  nonnegative  numbers  such  that 


(1.10) 


n  2 

Z  o  .  -  » 


n-1  „  k-1 


for  some  a  >  0.  Then  there  exists  an  array  {X^}  of  real  valued  r.v.'s 
such  that 


:>a 

<v**l 


(1.11)  e C Xnk u  -  o. 


a.12)  ECX^D-o  lk. 


(1.13)  ECXnkXmj3-0  (n  f  m  or  k  f  j ; 


k  ®  J  ®  1|2|«  i  in  s  n  i  in  ®  ly2i*»i)i 


c 
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(1.14)  lim  sup  |C  |  *  ® 

n-«o  n 


sointwise. 


Proof.  In  the  counterexample  we  will  construct,  the  underlying 

2 

probability  space  is  the  unit  square  I  =  [0,1)  *  [0,1)  with  the  Borel 
measurable  subsets  and  Lebesgue  measure. 

By  (1.10),  there  exists  a  sequence  (en)  of  positive  numbers  tending 
to  zero  such  that 


2 

e  n  9 

(1.15)  c  :  =  -S-  Z  o  .  <  1 
n  n2a  j=1  nj  - 


(n  =  1,2, ... ) 


(1.16) 


Z  c  =  <*>. 
n*l  n 


We  define 


,  2 

n-1  En  k  2 

d  .  ;  ■  E  c  +  -y  E  o  .  ( k  *  l,2,...,n;  n  =  1,2,...) 

nk  m-1  m  n2a  j-1  nj 


where  we  mean  E  *0  in  case  n*l.  Denote  by  [  ♦  ]  the  greatest  integer 
m»l 


part.  Define  the  function  f^u^)  by 


(1.17)  fnk(u,l):  "  7"  (,V* 

nk  1  en  Jnk  1 


e  [0,  1 ) 


where  Ij  means  the  indicator  of  the  set  Jnk  and  where  Jnk  =  [dn^kl  - 
nk 

'Vk-l1’  dn.k  '  ldn.kn  uhe"  tdn,k-l>  *  ldn,k>  and  Jnk  "  [0>  dnk'ldnk1)u 
(dn,k-l  ‘  Idn,k-l1>1)  ”hen  ldn,k-l>  +  ldn,k>-  By  (1'15)  and  (1-‘7>> 


(1.18) 


4,  fnk  (“l)  "  °nk  * 


(1.19) 


ECf 


nk 


fnk(ul)  fnj(a)l)du,l  =  0 


(k  +  j). 


We  will  apply  the  Rademacher  functions  {rR  (w 2 ):  n  *  1,2,...} 
defined  by 


r  (w_) : 
n  2 


2 

Z 

k=l 


C-i)k_1i 


) , 

[  (k-1  )/2n,  k/2n)  L 


w2  e  [0,1). 


Obviously,  {r^}  is  a  sequence  of  independent,  identically  distributed 
r .v. ' s  with 


(1.20)  ECrn3  =  o  and  E[V3=1  (n-1,2,...). 

The  role  of  the  Rademacher  functions  in  the  theory  of  Banach  spaces  is 
well-known  (see,  e.g.,  Schwartz  (1981)). 

Finally,  we  set 

Xnk^wl,u,2^s  “  ^nk^l^n^01^  =  n  *  1,2,...). 

By  construction,  {fn^}  and  {rn>  are  independent  of  each  other.  Thus, 
by  (1.18)  -  (1.20), 

-ECfnk3  E  Crn3  -0, 

E  CrE]  -<&. 

ECV„j3  -ECf„kfnj3  *  CrED  -0  (k  +  J). 

ECEnWXn1D  -  ECf  tf  ,3  E  [trl  -0  (»  +  «>, 
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which  in  turn  provide  relations  (1.11)  -  (1.13). 


By  (1.15)  and  (1.16),  each  e  [0,1)  is  in  J  for  infinitely  many 


n,  but  for  exactly  one  k  when  it  is.  Thus,  for  each  (u^,  u^)  e  l4 


lc"'  ’  '7  A  x"“  (v  “2>! 


1  n  1 

=  —  |  —  rn(uj2) |  =  “  infinitely  often, 
n  n  n 


whence  (1.14)  follows. 


§2.  Extensions 


A)  We  can  consider  the  following  generalized  array  of  r.v.'s 


xu*  x12,  ....  X, 


X21 *  X22’  X2p^ 


X  . ,  X  - . X 

nl*  n2  np. 


where  (p  :  n  *  1,2,...)  is  a  not  necessarily  increasing  sequence  of 


positive  integers.  In  case  pn  *  n  for  all  n,  we  get  an  ordinary  (triangular) 


array. 


B)  We  can  substitute  any  sequence  (A(n):  n  =  1,2,...)  of  positive 


numbers  for  n°  in  definition  (1.3). 


’■  ■’  ‘  •  '  *  '  '  J.  1  .**  4 


C)  We  can  require  only  quasi-orthogonality  (see,  e.g.  Moricz  (1977)) 
instead  of  orthogonality.  More  exactly,  we  assume  the  fulfillment  of  (1.1), 


but  instead  of  (1.2)  we  only  need  the  existence  of  a  generalized  array 
{pn^:  k  *  l,2,...,p  ;  n  =  1,2,...}  of  nonnegative  numbers  such  that 


(2.1) 


E 

k-1 


pnk  -  C 


(n  =  1,2, ... ) 


where  the  constant  C  does  not  depend  on  n,  and 


(2.2) 


I E  C  (  X 


nk’ 


V31  ^ 


3n,  | k- j  j-t-1 


nk 


o  . 
nj 


(k,j  *  1,2,. ..,p;  n  -  1,2,...). 

In  the  particular  case  when  p^  =  1  and  p  ^  *  0  otherwise,  we  get  ordinary 
orthogonality. 

It  is  known  (see  again  Moricz  (1977))  that  (2.1)  and  (2.2)  imply 

that 

P  P 

n  2  n  7 

ECU  s  x  .  »  3  <  (1  +  2C)  E  a  .  (n  -  1,2,...). 
k-1  nlt  k-1  nk 

Now  the  fulfillment  of  this  moment  inequality  is  crucial  in  the  proof 
of  Theorem  1 . 

To  sum  up,  the  following  theorem  can  be  proved  along  the  same  lines 
as  Theorem  1. 

Theorem  1A.  Let  {X^:  k  -  l,2,...,p  ;  n  -  1,2,...}  be  a  generalized 
array  in  a  separable  Hilbert  space  H  satisfying  conditions  (1.1),  (2.1) 
and  (2.2),  and  let  (A(n):  n  -  1,2,...)  be  a  sequence  of  positive  numbers. 


I 

e?Ji 


(2.3) 


Z  — —  Z  a  .  <  », 

n=l  x2(n)  k=l  nk 


(2.4) 


lim  — — r  Z  X  ,  *  0  completely 
n^x.  Hn)  k=1  nk  - K - 1 


Even  in  this  very  general  setting,  condition  (2.3)  is  the  best 
possible  one  to  ensure  (2.4).  The  way  we  proved  Theorem  2  makes  it  possible 
to  prove  the  following  more  general  theorem. 

Theorem  2A.  Let  {o  .  :  k  =  l,2,...,p  ;  n  =  1,2,...}  be  a  generalized 
arrav  of  nonnegative  numbers  such  that 


"  1  n  9 

Z  ~ —  Z  a~  =  «. 
n-1  \Z(n)  k-1  nk 


Then  there  exists  a  generalized  arrav  iX  .  }  of  real  valued  r.v.'s  such  that 


conditions  (1.11)  and  (2.12)  are  satisfied. 


ECX  . X  .  ]  -  0 
nk  mj 


(n+morkfj; 


k  *  1 , 2 , . . . , p  ;  j  *  1,2,. ...p  ;  n,m  *  1,2,...), 
n  m 


1  im  sup  |  — r  Z  X  ,  | 

n-  A(n)  k-1  nk 


)Ointwise . 


$3.  Banach  Space  Valued  Random  Variables 


Many  authors  have  contributed  to  the  development  of  the  theory  of 
Banach  space  valued  r.v.'s.  However,  we  will  only  need  reference  to  the 


’  ’  *'  .  V  1  i*  }  •  V  •  1  "**  /  *  V  *•  |*  4*  *•  '**<  I  V  V,  Vv  4* 


V 


type  p  spaces  of  Hoffmann- Jprgensen  and  Pisier  (1976),  and  the  orthogonality 


moment  inequalities  of  Howell  and  Taylor  (1981).  Additional  details  on 


basic  properties  and  measurability  considerations  are  available  in  the 


literature  cited  there. 


Orthogonality  in  the  general  Banach  space  situation  becomes  much 


more  problematic  as  the  following  example  illustrates. 


Example  1.  Let  1  <  q  <  00  and  fcq  denote  the  set  of  sequences 


(x^:  k  *  1,2,...)  of  real  numbers  such  that 


ll(xk)ll:  =  (  Z  |xk|q)1/q  < 


It  is  well-known  that  £.q  with  the  usual  vector  operation  and  the  norm 


defined  above  is  a  separable  Banach  space.  Setting 


Xnk!  =  (0,...,0,rk,0,...)  (k  =*  1,2, ...,n;  n  -  1,2,...) 


where  the  (rk>  are  the  Rademacher  functions,  we  get  an  array  (X  k>  con¬ 


sisting  of  rowwise  independent,  zero  mean  r.v.'s.  So,  they  should  be 


rowwise  orthogonal  with  respect  to  any  reasonable  definition  of  ortho¬ 


gonality.  However,  in  trying  to  develop  (1.6),  we  observe  that 


ECU  2  Xn.  II 2 □  -  EC(  S  |r.  |q)2/q3 


k=l  ** 


while 


£  EC«nk'  3 


Thus,  for  1  £  q  <  2  the  inequality 


.2  _n‘ 


n  1 


EC'^  k?l  X«k'  3  *  “ST  ’  kJ,Et:,Xnk'  ^ 

n  n  n  n  k*l 


* 


11 


goes  in  the  wrong  direction,  and  thus  we  are  unable  to  duplicate  the 
results  of  Section  1. 

Hoffmann- Jorgensen  and  Pisier  (1976)  addressed  this  problem  by 
defining  a  separable  Banach  space  B  to  be  of  type  p,  l<p  <  2,  if  for 
every  sequence  {X^:  k  *  1,2,...}  of  independent,  zero  mean  r.v.'s  in 
B  with 

(3.D  ECnx^l  <  -, 

and  for  every  n  >  1,  we  have 

n  n 

(3.2)  EC#  £  X,#p]  <  C  S  E  C  HX.  «p  3 

k-1  R  “  k-1  * 

where  C  is  a  constant  not  depending  on  n  and  {X^} . 

Clearly,  every  separable  Banach  space  is  of  type  1  and  every  separable 
Hilbert  space  is  of  type  2  even  with  equality  holding  in  (3.2)  for  C  ■  1. 
Moreover,  the  £p  spaces  are  at  most  of  type  min  (2,p),  1  ^  p  <  <*>. 

Howell  and  Taylor  (1981)  used  James  type  orthogonality  in  defining 
a  sequence  {X^}  of  r.v.'s  in  a  separable  Banach  space  B  to  be  orthogonal 
in  LP(B),  1  <  p  <  »°,  if  (3.1)  is  satisfied  for  all  k  and 

n  n+m 

(3.3)  ECl^  a„(k)  Vh'P3  i  E  *.(«)  X.(k>'P] 

for  all  sequences  {a^}  of  real  numbers,  for  all  permutations  it  of  the 

positive  integers  (1,2 . m+n},  and  for  all  n  and  m. 

It  is  clear  that  orthogonal  r.v.'s  in  a  separable  Hilbert  space 
satisfy  (3.1)  and  (3.3)  with  p  ■  2.  It  is  not  hard  to  see  that  the 


12 


r.v.'s  defined  in  2,^  in  Example  1  satisfy  (3.1)  and  (3.3)  for  any  p, 
1  £  p<  2.  Note,  q  need  not  equal  p. 


Remark  2.  Proposition  2.1  of  Howell  and  Taylor  (1981)  states  that  a 
separable  Banach  space  B  is  of  type  p  if  and  only  if  (3.2)  is  satisfied 
for  all  n  and  for  all  sequences  {X^}  which  are  orthogonal  in  LP(B). 

Inequality  (3.2)  was  actually  established  for  the  weaker  concept 
of  unconditional  semi-basic  (ucsb)  r.v.'s  but  in  this  case  the  constant 
C  depends  on  the  particular  ucsb  sequence  {X^} .  On  the  other  hand,  in  the 
case  of  orthogonal  r.v.'s  the  constant  C  depends  only  on  the  space  B. 

To  illustrate  the  generality  of  the  concept  of  orthogonality  in 
Banach  spaces  we  present  the  following 

Example  2.  Let  1  <  p  <  2  and  let  Y  be  a  Borel  measurable,  real  valued 
function  defined  on  [0,1)  such  that 

r1 

EC  |Y|P]  -  [  |Y(cu1)|P  d«l  <  « 

but 

ECY23  -  Y2(w1)dw1  =  ~. 

Define  for  (w^,  ui^)  e  [0,1)  *  [0,1)  *  I2 


X  (w. ,  w_):  *  Y(ui. )  r  (w-)  (n  ■  1,2,...) 
n  1  L  l  n  L 

where  the  {r  }  are  the  Rademacher  functions. 

1  n' 

Then  the  r.v.’s  (Xn>  are  not  orthogonal  in  the  classical  sense  since 

E[X  X  ]  is  not  defined.  Indeed, 
n  re 


EC|XnXm|3  -ECY2|rnrm|3  -EQY2]  - 
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On  the  other  hand,  the  r.v.'s  {Xr}  are  James  type  orthogonal  in  LP(I2). 
This  is  a  consequence  of  the  fact  that  the  r.v.'s  {r^}  are  independent  and 
have  zero  mean,  and  therefore 

n  _ 

Vu  xi,(k)'P3 

-ECIVI'1]  E  C.JiS(k) 

_  n+m 

iEC|Y|»]  E  Clfcr  »„(k)  r,(k)1»3 
n+m 

-  EC  S  a  .  X  . 

k=l  if(k)  J 

Now  if  (X^s  k  *  l,2,...,n;  n  =  1,2,...}  is  an  array  of  rowwise 
orthogonal  (in  LP(B))  r.v.'s  in  a  Banach  space  B  of  type  p,  then  it 
follows  from  (3.2)  that 


S  E  H  l|-=— • 
n*l  n 


n 

2  X  .11 
k=l  nk 


c  n 

—  2  E  £  IIX 
k=l 


nk 


and  the  following  form  of  Theorem  1  is  obtained. 

Theorem  IB.  Let  {Xnk}  be  an  array  of  rowwise  orthogonal  (in  LP(B)) 
r.v.'s  in  a  Banach  space  B  of  type  p  for  some  1  <  p  <  2.  If 

(3.4)  s  _L>  z  EC«X.IIP3<» 

n=l  nP*  k-1  nk 

for  some  a  >  0,  then  we  have  (1.5). 

Remark  3.  Since  the  real  line  R  is  of  type  p  for  each  1  <  p  <  2, 
Theorem  IB  extends  both  the  concept  of  orthogonality  for  real  valued  r.v.'s 
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and  the  range  of  applicable  moment  conditions  which  yield  (1.5). 

Sufficient  condition  (3.4)  is  also  best  possible  even  for  real  valued 
r.v.'s  as  the  next  theorem  shows. 

Theorem  2B.  Let  {t  be  an  array  of  nonnegative  real  numbers  such 

that 

£  _1_  S  TP  -  . 

n=l  npa  k*l  nk 

for  some  1  <  p  <  2  and  some  a  >  0.  Then  there  exists  an  array  {X^}  of 
real  valued  r.v.'s  such  that  conditions.  (1.11)  and 

Eci*nklp:i 


are  satisfied,  they  are  rowwise  orthogonal  in  Lp(l^)  and  we  have  (1.14). 


For  the  proof  of  Theorem  2B,  replace  (1.15)  by 


en  n  o 

c!  *  —  £  <.  <  1  (n  =  1,2,...) 

n  npa  j-1  nj  - 


and  follow  the  steps  in  the  proof  of  Theorem  2  with  d  .  :  *  Z?  J  c. 

£p  k  nk  ‘”l  1 

+  52  xp..  The  rowwise  orthogonality  in  LP(IZ)  follows  since 

npa  J-l  nj 

J  . . J  „  are  disjoint  subsets.  It  also  follows  that  X  . . X 

nl  nn  nl  nn 

are  orthogonal  in  the  usual  sense.  Banach  space  versions  of  Corollary  1 


and  Theorems  1A  and  IB  are  also  available  with  similar  conditions. 
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